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I.  SUMMARY 


Vision:  To  address  the  main  concerns  of  the  Taskable  Agent  Software  Kit  (TASK)  por¬ 
tion  of  DARPA’s  Agent-Based  Computing  Program  we  developed  fundamental  theory  and 
novel  algorithms  for  adaptive  learning  in  autonomous  collective-agent  systems.  There  were 
two  major  goals.  The  first  was  a  mathematical  framework  for  analyzing  the  dynamical 
mechanisms  that  support  learning  in  a  range  of  novel  information  processing  substrates. 
The  second  was  adapting  this  single-agent  learning  theory  and  associated  learning  algo¬ 
rithms  to  the  distributed  setting  in  which  a  population  of  autonomous  agents  communicate 
to  achieve  a  desired  group  task.  The  results  provide  a  mathematically  sound  basis  for  quan¬ 
titatively  measuring  (i)  the  degree  of  individual-agent  intelligence,  (ii)  the  significance  of 
agent-environment  interaction,  and  (iii)  the  emergence  (or  lack  thereof)  of  cooperation  in 
agent  collectives. 

Innovative  Ideas  and  Tools:  A  novel  feature  of  the  projects  is  that  the  theoretical 
approach  synthesized  recent  results  in  the  areas  of  machine  learning,  statistical  inference, 
statistical  mechanics,  nonlinear  dynamics,  pattern  formation  theory,  and  the  developing 
area  of  complex  systems  and  then  applied  them  to  (i)  single-agent  learning  and  (ii)  adaptive 
learning  in  agent  collectives.  The  project  also  had  unique  and  regular  access  to  experts  in  a 
wide  range  of  disciplines — such  as,  biology,  economics,  cognitive  science,  and  sociology — that 
historically  have  investigated  collective  behavior.  The  phenomena,  insights,  and  results  in 
these  disciplines  was  particularly  helpful  in  grounding  the  project’s  approach  to  fundamental 
theory. 

Central  Hypothesis  and  Experiments:  The  first  part  of  our  central  working  hy¬ 
pothesis  was  that  intelligent  behavior  of  autonomous  agents  requires  them  (individually)  to 
adaptively  estimate  the  complexity  of  their  environment  (including  other  agents)  and  build 
internal  dynamical  models  that  capture  the  patterns  and  regularities  which  they  detect.  The 
second  part  of  the  hypothesis  was  that  measures  of  cooperation  in  agent  collectives  can  be 
built  out  of  the  theory  describing  individual  agent  capabilities. 

The  basic  effort  established  the  mathematical  foundations  of  what  constitutes  useful  pat¬ 
terns  and  regularities  in  environmental  stimuli  and  how  these  can  be  adaptively  modeled 
and  used  as  a  basis  for  action  and  decision-making  by  autonomous  agents.  The  hypothe¬ 
sis  was  evaluated  by  developing  mathematical  models,  rigorously  bounding  their  minimal 
information  processing  and  computational  properties,  developing  analytical  approximations 
that  predict  single-agent  and  agent-collective  behavior,  and  comparing  these  results  with 
extensive  simulations  for  refining  the  mathematical  framework  and  for  final  validation  the 
approach.  We  developed  an  experimental  distributed  robotic  platform  to  test  these  results. 

Expected  Impact:  A  sizable  literature  exists  on  agent-based  systems.  [1-14]  Given  this 
and  the  diversity  of  proposals  for  designing  and  implementing  intelligent  agents  and  the 
wide  range  of  collective  behaviors  exhibited  by  interacting  agents,  [15-47]  we  believe  the 
central  impact  of  the  project  will  be  its  systematic  and  quantitative  approach  to  predicting 
behavioral  complexity  in  agents  and  in  their  environments.  This  will  provide  a  foundation  for 
measuring  agent  and  agent-collective  intelligence  which,  in  turn,  should  allow  for  systematic 
engineering  and  monitoring  of  these  systems. 
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II.  INTRODUCTION 


This  project  addressed  two  questions:  How  can  we  design  agent  collectives  to  achieve  a 
desired  global  function?  And,  how  can  we  do  this  so  that  the  collective  behavior  is  robust? 
To  address  various  aspects  of  these  questions,  we  focused  on  several  projects  whose  overall 
intent  was  to  develop  novel  algorithms  for  adaptive  learning  in  autonomous  agent  systems. 
There  were  two  major  goals.  The  first  was  developing  a  new  dynamical  framework  for 
analyzing  the  nonlinear  processes  that  support  learning  and  adaptation  in  individual  agents. 
The  second  was  generalizing  this  single-agent  learning  theory  to  the  distributed  setting  in 
which  a  population  of  agents  communicate  to  achieve  a  group  task. 

A  practical  objective  in  these  was  to  develop  methods  for  detecting,  monitoring,  and 
quantifying  the  degree  of  cooperation  that  emerges  in  interacting  multiagent  systems.  An¬ 
other  was  to  develop  algorithms  that  would  allow  individual  adaptive  agents  to  build  models 
of,  and  so  come  to  predict,  their  environments. 

A  brief  chronology  will  help  show  how  the  project  addressed  these  goals  over  its  lifetime. 

From  its  start  in  June  2000  to  July  2001,  the  project  was  involved  in  establishing  the 
Dynamics  of  Learning  group  at  the  Santa  Fe  Institute.  The  primary  research  focus  was  on 
single-agent  learning  and  adapting  methods  from  computational  mechanics  to  that  problem. 

From  August  2001  to  July  2002,  a  bridge  was  established  between  the  dynamics  of  single- 
agent  learning  and  the  notions  of  randomness  and  structural  complexity  from  computational 
mechanics.  In  particular,  the  outlines  of  an  algorithm  to  incrementally  infer  causal  states 
was  worked  out.  The  algorithm  was  a  key  component  in  our  analysis  of  single-agent  learning. 
In  short,  the  goal  of  an  agent’s  adaptive  dynamics  is  to  infer  a  causal  model  (e-machine,  see 
below)  of  its  environment. 

From  August  2002  to  July  2003,  the  project  investigated  how  complicated  a  multiagent 
system’s  dynamics  can  be.  We  developed  new  tools,  both  analytical  and  software,  for  explor¬ 
ing  this  kind  of  behavioral  diversity  and  showed  that  the  full  range  of  nonlinear  behaviors 
from  stable  oscillations  to  deterministic  chaos  can  occur.  We  also  established  a  modeling 
framework  that  allowed  us  to  analyze  the  mechanisms  that  lead  to  such  behaviors. 

From  August  July  2003  to  July  2004,  the  key  effort  was  to  find  methods  to  grapple 
with  the  complicated  behaviors  that  multiagent  system’s  exhibit.  In  addition  to  developing 
a  dynamical  systems  theory  analysis  for  these,  a  central  part  was  the  development  of  our 
Robotic  MultiAgent  Development  System  (RoMADS)  experimental  platform.  RoMADS 
allows  for  real,  physical  experimentation  with  robotic  vehicle  collectives. 

From  August  2004  to  December  2005,  the  project’s  termination,  the  focus  was  on  the 
TASK  Demonstration  project  and  developing  a  new  version  of  the  RoMADS  system.  We 
also  focused  on  innovations  in  the  dynamical  systems  modeling  of  interacting  agents  that 
allow  one  to  study  the  dynamical  stability  (and  instability)  of  a  much  wider  range  of  agent 
adaptation  algorithms  than  previously. 
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A.  Dynamics  of  Learning 


Some  time  ago,  we  introduced  an  approach,  called  computational  mechanics,  [48,  49]  for 
detecting  information  processing  capabilities  embedded  in  nonlinear  processes.  Computa¬ 
tional  mechanics  methods  allow  one  to  delineate  (i)  how  much  historical  information  is  stored 
in  a  process’s  current  state,  (ii)  how  this  memory  is  organized,  and  (iii)  how  this  organiza¬ 
tion  produces  future  behavior.  We  refer  to  these  three  properties  as  a  process’s  intrinsic 
computation. 

An  important  component  of  computational  mechanics  is  its  constructive  approach  to  in¬ 
trinsic  computation:  it  provides  algorithms  for  estimating  intrinsic  computational  properties. 
This  leads  naturally  to  the  present  project  in  that  these  algorithms,  and  the  theory  describing 
them,  give  a  principled  approach  to  learning  optimal  models  of  nonlinear  processes. 

Thus,  the  central  approach  was  to  use  computational  mechanics  to  develop  a  theory  of 
how  a  dynamical  system,  as  part  of  its  behavior,  can  learn  a  model  of  its  “environment” ;  that 
is,  of  signals  imposed  on  it.  This  type  of  dynamical  learner  was  our  model  for  the  processes 
controlling  intelligent  agents.  One  of  the  key  ideas  here  is  that  of  intrinsic  representations 
introduced  by  us.  [48,  49]  The  learning  theory  we  developed  showed  how  an  agent  can  infer 
these  representations  and  use  them  to  predict  the  environment’s  future  behavior  and,  in  this 
way,  to  take  advantage  of  environmental  regularities. 

It  is  important  to  emphasize  that  learning  in  a  natural  environment  is  not  like  the  learning 
of  language  by  humans.  The  latter  is  certainly  an  important  topic;  it  provides  many  of  the 
metaphors  for  learning  in  artificial  intelligence.  Learning  in  a  natural  environment,  however, 
is  not  constrained  by  a  human  language  syntax  nor  even  by  the  need  for  symbolic  codification. 
Therefore,  learning  in  a  natural  environment  cannot  rely  on  innate  or  required  grammatical 
or  other  linguistic  constraints.  In  fact,  the  analogues  of  such  constraints,  if  they  exist,  must 
be  learned  from  the  environment.  These  constraints  reflect  real-world  geometry,  physical 
law,  and  a  wide-range  of  nonlinear  pattern-forming  processes  that  nature  expresses. 

We  believe  that  analogies  with  natural  language  acquisition  have  led  to  the  wrong  concep¬ 
tual  and  mathematical  framework  for  (i)  learning  in  natural  environments  and  (ii)  distributed 
adaptation  as  expressed  by  collectives  of  adaptive  agents.  These  problems  are  one  reason 
we  took  an  alternative  approach  to  learning  and  adaptation.  Specifically,  computational 
mechanics  does  not  suffer  from  the  problems  inherent  in  symbolic  approaches  to  intelligent 
behavior  and  its  design. 


B.  Emergence  of  Distributed  Adaptation 

Once  the  basic  theory  of  dynamical  learning  was  fleshed  out,  we  began — in  a  princi¬ 
pled,  analytical,  and  quantitative  way — to  investigate  the  interaction  of  individual  learners 
in  various  types  of  structured  populations.  We  refer  to  these  as  agent  collectives.  This 
approach — first  focusing  on  individual  learning  and  then  collective  dynamics — greatly  re¬ 
duced  the  number  of  parameters  (and  conceptual  degrees  of  freedom)  so  that  we  could 
cleanly  identify  those  problems  and  properties  that  are  the  result  only  of  collective  inter¬ 
actions.  Out  goal  was,  after  all,  to  determine  what  agent  collectives  can  do  that  is  novel, 
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unique,  and  ultimately  better  (or  worse)  than  other  alternatives  to  performing  a  given  task. 

To  be  more  explicit,  using  the  computational  mechanics  approach  we  addressed  the  fol¬ 
lowing  three  questions. 

1.  What  is  “distributed”  adaptation? 

2.  How  can  we  detect  and  design  the  “emergence”  of  distributed  adaptation? 

3.  What  is  a  true  “collective”?  Does  the  group  do  more  than  the  sum  of  the  individual 
agents? 

The  following  sections  briefly  expand  on  these  questions  so  that  we  can  detail  onr  chosen 
projects  on  the  dynamics  of  learning  and  the  emergence  of  distributed  adaptation  and  their 
results.  The  next  section  addresses  how  onr  approach  is  different  from  and  also  similar  to 
current  approaches  to  learning,  on  the  one  hand,  and  distributed  artificial  intelligence,  on 
the  other.  The  report  then  gives  a  synopsis  of  the  individual  problems  and  results  developed 
during  the  project.  It  concludes  with  a  few  general  remarks.  The  appendices  provide  lists 
of  workshops  the  project  organized,  the  publications  produced,  and  the  presentations  given 
by  group  members  on  the  project.  Finally,  the  last  appendix  gives  a  list  of  visiting  scholars’ 
presentations  to  the  Dynamics  of  Learning  group. 


III.  METHODS,  ASSUMPTIONS,  AND  PROCEDURES 

Our  approach  used  concepts  and  analytical  tools  from  statistical  and  nonlinear  physics, 
theoretical  computer  science,  and  communication  theory.  What  is  novel  about  the  approach 
was  the  integration  we  achieved  of  these  disparate  fields  for  the  problem  of  detecting  adap¬ 
tation  in  individual  agents  and  cooperation  in  multiagent  systems. 

In  the  past,  detecting  emergent  cooperation  has  been  done  subjectively.  Our  goal  was 
to  understand  the  phenomenon  well  enough  so  that  we  could  develop  analytical  and  predic¬ 
tive  theories  of  emergent  cooperation  and  algorithms  that  automatically  (and  objectively) 
quantify  and  monitor  cooperation. 

We  believe  that  new  design  methods  for  multiagent  systems  will  follow  directly  from  onr 
results.  Moreover,  we  believe  that  our  results  will  lead  to  diagnostic  tools  for  large-scale 
distributed  systems  that  consist  of  semi-autonomous  intelligent  agents. 

The  wide  applicability  of  onr  approach  is  one  of  its  most  promising  aspects.  First,  the 
basic  problem  of  emergent  cooperation  occurs  in  many  natural  and  artificial  systems.  Second, 
the  results  should  be  applicable  to  autonomous  robotic  vehicles,  for  example,  as  well  as  to 
software  agents  on  the  web. 


A.  Brief  Overview  of  the  Approach 

Having  sketched  the  basic  problems,  it  is  worthwhile  to  outline  what  we  saw  as  the 
distinctive  concepts  underlying  onr  approach. 
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1.  Informational  complexity  of  learning :  We  developed,  using  recent  results  in  machine 
learning  and  computational  learning  theory, [3,  12,  13,  50-53]  a  theory  that  describes 
the  constraints  on  and  requirements  for  learning  models  of  environments  with  varying 
degrees  of  complexity. 

2.  Dynamical  setting :  We  adapted  these  results  and  the  theory  of  trade-offs  into  a  dy¬ 
namical  setting  to  develop  algorithms  for  time-dependent,  incremental,  and  adaptive 
learning. 

3.  Distributed  adaptation:  We  then  extended  the  analysis  of  learning  dynamics  to  a  dis¬ 
tributed  setting,  in  which  collections  of  largely  autonomous  adaptive  agents  interact 
via  some  communication  network. 

4.  Evolutionary  and  ecological  setting:[ 9,  14,  15,  17,  35,  50,  54-58]  We  merged  these  results 
with  evolutionary  adaptation  by  developing  the  appropriate  mathematical  population 
dynamics.  In  particular,  we  investigated  these  results  for  finite  collections  of  interacting 
finite-memory  agents. 


B.  Relationship  to  Existing  Theory 

The  project  was  only  possible  due  to  recent  advances  in  several  fields.  Computational 
mechanics  borrows  heavily  from  several  areas.  To  get  a  sense  of  the  overall  approach,  it  is 
useful  to  briefly  mention  several  of  these  fields  and  what  they  contributed. 

1.  Dynamical  systems  theory:  [59-61]  Geometric  analysis  of  state-space  structures  that 
guide  nonlinear  systems  in  their  temporal  behavior;  the  control  of  chaotic  dynamical 
systems  using  very  small  signals  to  manipulate  and  suppress  inherent  instabilities.  [62] 

2.  Statistical  mechanics: [63-66]  Models  of  how  macroscopic  behavior  emerges  from  mi¬ 
croscopic  equations  of  motion.  Can  we  detect  and  then  characterize,  at  a  “thermo¬ 
dynamic”  level,  collective  (and,  hopefully,  useful)  behavior  in  populations  of  adaptive 
agents? 

3.  Modern  statistical  inference[ 67,  68]  for  nonlinear  processes:  Trade-offs  between  sample 
size  and  model  complexity.  This,  to  mention  just  one  example,  has  not  been  extended 
to  spatial  or  distributed  collections  of  adaptive  agents. 

4.  Pattern  formation  theory,  phase  transitions,  and  critical  phenomena:[Q9 ,  70]  How  dy¬ 
namical  and  statistical  ensemble  mechanisms  lead  to  cooperative  behavior. 

5.  Computational  learning  theory ,  including  aspects  of  machine  learning  and  AI:[3,  12, 
13,  50-53,  55,  71]  Recent  progress  in  understanding  the  resource  trade-offs  in  various 
types  of  learning  paradigm:  artificial  neural  networks,  hidden  Markov  models,  and 
more  abstract  settings,  such  as  Valiant’s  PAC  learning,  [72]  but  within  distribution- 
dependent  learning  paradigms. 
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Information  theory[73]  and  computation  theory[74,  75]  also  played  important  roles, 
though  largely  by  providing  basic  tools  and  observables  for  analysis. 


IV.  RESULTS  AND  DISCUSSION 

We  reformulated  and  refined  the  main  questions  given  above  in  the  following  ways — ways 
that  motivated  our  selection  of  subprojects. 

1.  How  little  intelligence  can  autonomous  agents  use  and  still  achieve  a  given  level  of 
functionality — individually  and  collectively? 

2.  What  additional,  global  capabilities  (if  any)  do  agent  collectives  have? 

3.  Does  global  capability  allow  individual  agents  to  be  simpler  still? 

4.  How  do  we  detect  (quantitatively)  that  a  given  designed  collective  is,  in  fact,  more 
functional  than  another? 

5.  What  is  the  balance  of  local  functionality  and  distributed  cooperation? 

6.  How  much  flexibility  is  there  in  achieving  such  a  balance? 

7.  How  do  different  communication  topologies  affect  global  function  and  an  agent’s  min¬ 
imal  required  intelligence? 

8.  Can  we  predict  that  a  given  collective  is  more  functional,  adaptive,  or  robust  than 
another? 

For  each  question  and  related  subproject  a  number  of  ancillary,  but  common  issues  arise: 

1.  How  do  we  define  a  workable,  measurable  notion  of  adaptation? 

2.  How  do  we  define  a  workable,  measurable  notion  of  functionality? 

3.  How  do  we  define  a  workable,  measurable  notion  of  emergence? 

4.  What  observables  are  appropriate  for  experimental  detection  of  these  properties  and 

for  theory  development? 

5.  How  can  we  define  robustness  in  learning,  adaptation,  and  evolution? 

6.  What  are  the  resource  trade-offs  between,  e.g., 

(a)  Sample  size  versus  model  complexity, 

(b)  Response  time  versus  model  complexity, 

(c)  Collective  size  versus  global  function,  and 

(d)  Collective  size  versus  individual  intelligence? 
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We  selected  several  projects  to  address  various  aspects  of  the  above  questions  and  ancil¬ 
lary  issues.  The  chosen  projects  cannot  decisively  answer  these  questions,  if  the  latter  are 
construed  too  broadly.  With  this  and  the  size  of  the  overall  effort  in  mind,  we  focused  on  the 
following  subprojects  according  to  (i)  their  importance  to  the  main  questions,  (ii)  the  like¬ 
lihood  of  success  in  tackling  them,  and  (iii)  our  existing  expertise  and  available  computing 
resources.  These  were  grouped  into  two  broad  efforts,  labeled  Task  1  and  Task  2. 

Task  1.  For  the  dynamics  of  agent  learning  the  selected  projects  are  the  following: 

1.  Computational  Mechanics:  Foundations. 

2.  Computational  Mechanics:  Algorithms. 

3.  Single- Agent  Learning. 

4.  Discovering  Spatial  Patterns. 

5.  Finite-Data  Scaling  of  Pattern  Discovery. 

Task  2.  For  analyzing  the  emergence  of  distributed  adaptation  in  agent  collectives  the 
projects  are  the  following. 

1.  e-Machine  Nets  as  Models  of  Agent  Collectives. 

2.  Finite-Population  Dynamical  Systems. 

3.  Evolution  of  Finite-Memory  Agents. 

4.  Self- Organizing  Agent  Population  Dynamics. 

5.  Theory  and  Analysis  of  Emergent  Collective  Adaptation. 

6.  Persistent  Chaos  in  High  Dimensions. 

7.  Dynamics  of  Multiagent  Games. 

8.  Multiple  Agents  Servicing  Multiple  Tasks. 

9.  Causal  Synchrony. 

10.  Robotic  MultiAgent  Development  System  (RoMADS). 


A.  Task  1:  Dynamics  of  Learning 

We  continued  to  develop  our  multidisciplinary  approach  to  computation  in  dynamical 
systems  and  to  inferring  models  (specifically,  e-machines)  of  nonlinear  processes,  as  the 
central  mathematical  framework  for  the  dynamics  of  single-agent  learning.  We  refined  our 
existing  tools  in  ways  that  facilitated  our  analyzing  the  intrinsic  computation  in  distributed 
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dynamical  systems.  The  new  emphasis,  however,  was  to  understand  the  resource  trade¬ 
offs  in  changing  model  classes  to  produce  optimal  models  and  forecasts  from  temporal  and 
spatio-temporal  data  series. 

These  refinements  were  necessary  for  practical  reasons.  In  autonomous  agent  learning 
sensory  data  is  limited  in  quantity,  has  low  signal-to-noise  ratio,  and  is  nonstationary.  Thus, 
efficient  algorithms  for  e-machine  reconstruction  are  essential.  Developing  these  algorithms, 
in  turn,  requires  a  deeper  appreciation  of  the  dynamics  of  learning  and  adaptation.  Aside 
from  the  pragmatic  goals,  the  related  projects  elucidated  a  number  of  fundamental  resource 
trade-offs  present  in  learning  and  adaptation. 


1.  Computational  Mechanics:  Foundations 

The  basis  for  this  was  the  notion  of  an  optimal  representation  afforded  by  causal  states  and 
e-machines.  We  had  already  established  a  suite  of  theorems  that  provide  the  mathematical 
foundations  of  computational  mechanics.  [48,  76]  Specifically,  we  proved  that  as  one  ranges 
over  alternative  choices  of  “state”  the  e-machine  has  a  three-fold  optimality:  (i)  no  set  of 
alternative  states  is  more  informative  about  future  observables  than  the  causal  states;  of 
those  choices  of  state  that  are  as  predictive  as  the  causal  states,  none  has  (ii)  a  smaller 
statistical  complexity  nor  (iii)  a  smaller  entropy  rate  over  the  internal  states.  Much  of  the 
early  theoretical  development  was  concerned  with  continuing  exploration  and  refinement  of 
the  mathematical  foundations  of  computational  mechanics.  We  are  particularly  interested 
in  analyzing  the  effects  of  measurement  distortion  on  these  types  of  optimality. 

During  this  project  we  developed  means  of  explicitly  calculating  the  average-case  growth 
rates  of  most  standard  complexity  measures  (algorithmic  information,  thermodynamic  depth, 
logical  depth,  sophistication,  minimum  description  length,  and  the  like)  from  the  causal 
state  representation  of  a  complex  system.  This  work  provided  the  first  means  for  explicit 
calculation  of  most  of  these  measures  and  showed  that  many  of  them  are  asymptotically 
equivalent  (e.g.,  thermodynamic  depth  and  sophistication). 

We  successfully  extended  computational  mechanics  to  use  measure-theoretic 
foundations.  [77]  This  will  allow  for  a  much  wider  range  of  applications  of  our  notions 
of  causal  structure — for  example,  to  statistical  mixture  processes  and  continuous-valued 
processes. 


2.  Computational  Mechanics:  Algorithms 

There  were  a  number  of  new  algorithms  that  we  developed  as  we  adapted  computational 
mechanics  to  single-agent  and  multiagent  learning. 

a.  Agent  adaptation  through  incremental  learning  We  completed  our  new  incremental 
learning — Causal  State  Splitting  Reconstruction  (CSSR  or  “scissor”) — algorithm.  This  in¬ 
fers  causal  models  from  time  series  by  adaptively  splitting  candidate  causal  states.  We  proved 
its  convergence  on  the  optimal  model  using  a  combination  of  conditional- independence  and 
large-deviations  arguments.  We  also  performed  a  large-scale  Monte  Carlo  simulation  to 
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experimentally  demonstrate  the  algorithm’s  feasibility.  [78,  79] 

b.  Real-time  dynamical  learning  algorithms  The  goal  in  this  was  to  update  and  modify 
the  structure  of  an  estimated  e-machine  as  measurement  data  is  sensed — i.e.,  in  real-time. 
This  is  a  key  step  in  practical  implementations  of  e-machine  reconstruction  for  autonomous 
agents.  Additionally,  this  type  of  incremental  learning  formed  the  basis  of  our  studies  of 
adaptation  in  e-machine  nets,  which  are  described  below. 

With  an  incremental  version  of  e-machine  reconstruction,  a  number  of  questions  about 
adaptive  learning  arise  that  we  attempted  to  answer.  Outstanding  ones  were  the  following. 

1.  Is  incremental  learning  necessarily  faster  than  our  current  batch  reconstruction  meth¬ 
ods? 

2.  To  which  learning  problems  is  incremental  learning  best  adapted? 

3.  How  does  the  on-line  property  compromise  the  descriptive  capability  of  e-machines? 

4.  Is  incremental  learning  more  parsimonious  in  its  use  of  available  resources  and  input 
data? 

The  goal  was  to  embed  the  on-line  learning  algorithm  into  a  dynamical  system  itself,  so 
that  the  system’s  attractor  has  a  subspace  that  is  a  model  of  an  applied  signal.  One  important 
aspect  of  analyzing  such  a  real-time  dynamical  learner  is  the  need  for  a  formal  definition 
of  adaptation.  This  is  necessary  to  monitor  how  well  the  learning  is  being  performed,  as 
well  as  how  robust  it  is  to  fluctuations  and  nonstationarity.  Several  extensions  of  existing 
dynamical  systems  measures — e.g.,  conditional  Lyapunov  exponents  or  conditional  statistical 
complexity — were  explored  as  a  basis  for  a  quantitative  definition  of  adaptation  and  the  rate 
of  adaptation. 

c.  Hierarchical  e-machine  reconstruction  We  investigated  how  to  innovate  new  model 
classes  from  inadequate  ones.  Innovating  new  model  classes  had  been  worked  out  for 
the  transition  from  finite-state  machines  to  nested-stack  automata,  from  stochastic  finite- 
state  machines  to  infinite-state  stochastic  machines,  and  from  cellular  automata  to  cellular 
transducers.  [49]  This  subproject  was  reduced  to  coding  up  an  algorithm  that  facilitated  the 
innovation  of  hierarchical  model  classes.  Building  on  the  learning  resource-interdependence 
work,  we  investigated  the  trade-offs  between  representation  and  modeling  efficiency  and  the 
associated  resource  requirements  for  innovating  new  classes  of  representation. 

d.  Spectral  reconstruction  of  causal  architecture  We  developed  a  new  method — -the 
e-Machine  Spectral  Reconstruction  (eMSR  or  “emissary”)  algorithm — for  inferring  causal 
models  from  spectral  (rather  than  temporal)  data.  We  successfully  applied  this  to  experi¬ 
mental  scattering  spectra  from  complex  materials.  [80-83]  In  addition  to  working  on  the  the¬ 
oretical  basis,  we  extended  the  applications  beyond  solid-state  materials  to  spectral  sensors 
for  intelligent  agents.  Initial  results  on  solid-state  materials  have  been  extensively  expanded 
and  compared  with  established  models.  They  show  a  considerable  improvement  in  both 
predictive  value  and  ability  to  infer  physical  parameters. 
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3.  Single- Agent  Learning 


We  developed  a  theory  that  describes  the  consequences  of  an  agent’s  inability  to  model  a 
structured  environment:  to  the  extent  that  the  environment’s  structural  complexity  (mea¬ 
sured  quantitatively)  exceeds  the  representational  or  storage  capacity  of  the  learning  agent, 
then  the  environment  appears  more  random  (in  proportion  to  the  difference). 

a.  Information  Theory  of  Agent  Learning  We  adapted  our  previous  information- 
theoretic  characterization  of  the  structural  complexity  of  nonlinear  processes.  The  generality 
of  the  approach  allowed  us  to  analyze  a  number  of  learning  and  adaptation  problems  faced 
by  a  wide  range  of  agent  types  and  in  a  variety  of  environment — e.g.,  physical  robots  and 
software  agents.  [84] 

b.  Synchronizing  to  the  Environment  We  developed  a  rigorous  analysis  of  the  process  by 
which  an  agent  comes  to  know  the  state  of  its  environment.  [85]  In  this  work  we  introduced  a 
new  information  theoretic  quantity,  which  we  called  transient  information ,  that  measures  the 
total  uncertainty  experienced  while  an  agent  becomes  synchronized  to  its  environment.  This 
quantity  is  important,  compared  to  existing  information-theoretic  measures  of  environmental 
complexity,  in  the  situation  when  an  agent  must  act  immediately  on  incomplete  information 
about  its  environment.  We  applied  the  general  theory  to  a  widely  encountered  problem  in 
communications  and  control  system  design  in  multi-agent  systems:  how  to  synchronize  most 
efficiently  and  noise-tolerantly  to  a  periodic  signal.  We  developed  a  general  theory  that 
should  be  easily  applied  to  a  number  of  specific  problems  in  multi-agent  communication  and 
coordination. 


4-  Discovering  Spatial  Patterns 

We  made  significant  progress  on  computational  mechanics  for  multidimensional  infor¬ 
mation  sources.  [86,  87]  This  included  extending  the  formal  theory  of  epsilon-machines  and 
causal  states  to  arbitrary  dimensional  spaces  and,  also,  developing  and  validating  a  recon¬ 
struction  algorithm  that  can  build  spacetime  epsilon-machines  from  1+1  D  dimensional  data. 
This  established  our  conjecture  that  computational  mechanics  can  be  extended  to  multidi¬ 
mensional  data  in  a  constructive  manner.  The  results  are  useful  for  intelligent  agents  that 
must  infer  aspects  of  spatial  structure  in  their  environment. 

We  developed  methods  for  inferring  causal  models  from  multidimensional  spatiotemporal 
data.  These  consist  of  a  suite  of  computational  tools  for  automatically  generating  hypothe¬ 
ses  concerning  emergent  structures  in  spatial  systems  and  for  automatically  and  efficiently 
testing  such  hypotheses.  The  result  is  a  method  to  discover  novel  patterns,  not  seen  before. 
We  compared  the  methods  to  conventional  ones  of  statistical  mechanics  in  the  case  of  dy¬ 
namic  spin  systems.  We  reformulated  the  problem  of  identifying  patterns  in  two  and  higher 
dimensions  in  a  way  that  promises  to  overcome  many  prior  limitations.  [86] 

We  developed  new  algorithms  for  building  the  nonlinear  filters  (transducers)  that  are  used 
for  detecting  embedded  structures.  [87]  Having  committed  to  one  version  of  the  previously 
developed  formalism,  we  analyzed  examples  to  see  if  it  can  be  reduced  to  practice. 
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5.  Finite-Data  Scaling  of  Pattern  Discovery 


The  relationship  between  a  system’s  information  processing  characteristics — e.g.,  its  sta¬ 
tistical  complexity  and  entropy  rate — determine  in  large  measure  the  difficulty  of  learning  a 
model  of  it.  This  difficulty  is  directly  reflected  in  the  compute  time  and  space  and  in  the 
amount  of  data  required.  Our  goal  was  a  general  scaling  theory  of  these  trade-offs  along  the 
lines  found  is  phase  transition  theory.  [64,  88] 

We  extended  our  current  methods  to  study  the  effect  of  finite  resources  on  the  optimality 
of  reconstructed  e-machines.  This  is  an  important  problem  from  several  viewpoints.  The 
first  viewpoint  concerns  the  practical  problem  of  implementation  efficiency.  Given  fixed 
amounts  of  time  and  computational  resources  (instruction  set  and  storage),  what  level  of 
approximation  can  the  inference  technique  be  expected  to  attain?  The  second  viewpoint 
can  be  described  in  biological  terms:  organisms  during  their  evolution  and  development  are 
faced  with  exactly  the  same  trade-off  between  modeling,  predicting,  and  taking  advantage 
of  observed  regularities  in  environment  states.  These  trade-offs  arise  from  limited  metabolic 
resources  and  fixed  capacities  to  perceive  and  model  the  environment.  Analytic  results  on 
these  trade-offs  go  some  distance  toward  delineating  the  structural  and  information  process¬ 
ing  requirements  for  such  adaptive  organisms. 

The  main  questions  centered  on  how  to  balance  the  cost  of  additional  machine  components 
(causal  states  and  transitions)  against  improved  prediction  and  overfitting.  At  each  stage 
the  e-machine’s  structure  should  be  the  simplest  possible  so  that  the  available  data  yields  the 
best  statistical  predictions  of  environment  behavior  and  so  that  the  model  has  the  highest 
probability  of  generalizing  to  new  data.  Large  models,  in  contrast,  spread  the  given  data  more 
thinly  over  a  large  number  of  internal  parameters,  thereby  increasing  the  estimation  error. 
Simultaneously,  the  machine’s  ability  to  predict  future  observations  should  be  maximized. 

To  investigate  how  these  considerations  are  balanced,  we  measured  the  e-machine’s  struc¬ 
ture  via  the  statistical  complexity.  The  prediction  error  was  measured  by  the  length  of  the 
machine-encoded  data.  This  length  was  to  be  minimized  along  with  the  e-machine  com¬ 
plexity.  Our  optimality  criterion  was  a  version  of  various  minimum  modeling  principles  put 
forth  by  Rissanen,  [89]  Wallace,  [90]  and  others.  [91,  92]  (This  approach  should  not  be  con¬ 
fused,  however,  with  one-pass  data  compression  techniques,  such  as  Ziv-Lempcl  coding,  [93] 
in  which,  even  for  finite-state  sources,  the  effective  model  is  infinite. [89,  90]) 


B.  Task  2:  Emergence  of  Distributed  Adaptation 

In  large-scale  distributed  systems  the  elementary  components — agents,  robots,  neurons, 
ants,  firms,  and  so  on — behave  as  if  they  themselves  are  complicated  dynamical  processes. 
Many  of  their  important  and  defining  characteristics  are  the  product  of  nonlinear  interactions 
and  emergent  cooperative  behavior.  How  do  they  individually  and  collectively  process  infor¬ 
mation?  How  do  their  dynamical  behaviors  and  susceptibility  to  noise  interact  to  support 
and  limit  their  computational  capabilities? 

Our  goal  in  approaching  these  questions,  at  the  most  basic  level,  was  analogous  to  that 
found  in  Shannon’s  pioneering  work  in  developing  a  theory  of  “communication”  .[73,  94] 
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Shannon  explicitly  disavowed  using  his  mathematical  theory  to  analyze  the  meaning,  se¬ 
mantics,  utility,  and  information  content  carried  in  communication  channels.  Instead,  his 
theory  considered  only  the  amount  of  information  and,  due  to  this  restriction,  he  was  able  to 
derive  very  basic  constraints  on  reliable  information  transmission.  The  quantitative  aspects 
of  onr  computational  mechanical  approach  stand  in  the  same  relation  to  distributed  dynam¬ 
ical  systems  that  Shannon’s  theory  did  to  the  particular  domains  of  communication  that  a 
channel  can  carry.  At  this  level,  we  wished  to  understand  the  basic  constraints  between  the 
(statistical)  complexity  of  an  agent-agent  “communication  channel”  and  the  rate  of  reliable 
information  processing  of  agent  collectives.  More  importantly,  though,  the  computational 
mechanics  approach  is  distinct  from  Shannon’s  original  goals  in  that  computational  mechan¬ 
ics  explicitly  provides  a  procedure  for  detecting  the  effective  “codes”  or  “protocols”  that 
carry  the  significant  information  within  a  channel.  Furthermore,  it  differs  in  that  it  also 
relates  these  codes  to  (i)  known  types  of  computation,  (ii)  state  space  structures  that  guide 
the  dynamical  behavior  of  the  underlying  system,  and  (iii)  semantic  and  functional  aspects 
of  those  structures. 


1.  e-Machine  Nets  as  Models  of  Agent  Collectives 

Our  investigation  of  how  individual  agents  and  agent  collectives  perform  computations 
involved  analyzing  simulations  of  related  nonlinear  dynamical  systems  and  of  a  new  type  of 
e-machine-based  distributed  model — called  an  e-machine  net. 

Underlying  the  problems  and  themes  noted  above,  one  discerns  a  common  theoretical 
model:  finite-state  or  infinite-state  stochastic  e-machines  with  input  sensors  and  output 
effectors.  We  refer  to  these  as  e-transducers.  They  are  the  mathematical  articulation  of 
the  model  implied  by  the  computational-mechanical  analysis  of  a  single  learning  agent.  In 
particular,  e-transducers  are  defined  so  as  to  capture  all  of  the  information  processing  and 
computing  that  is  embedded  in  a  learning  agent.  The  goal,  of  course,  is  that  individual 
agents  be  adaptive,  in  the  senses  discussed  above.  This  is  naturally  modeled  within  the 
computational  mechanics  framework  by  including,  in  one  form  or  another,  the  e-machine 
reconstruction  algorithm  as  part  of  the  internal  agent  control  dynamics.  In  this  way,  adaptive 
agents  can  be  represented  by  e-machine  learners.  As  it  reads  its  input  stimulus,  each  e- 
transducer  learner  changes  structure  according  to  a  real-time  dynamical  version  of  e- machine 
reconstruction,  as  described  above.  The  net  result  is  a  purely  computation-  and  information- 
theoretic  model  of  the  active,  adaptive  processing  performed  by  an  agent.  Thus,  agent 
collectives  can  be  modeled  as  a  network  of  interacting  e-transducers;  a  model  that  we  refer  to 
as  an  e-machine  net.  This  is  a  collection  of  e-transducers  with  inputs  and  outputs  connected 
in  a  fixed  or  possibly  dynamic  communication  topology.  The  global,  adaptive  behavior  that 
emerges  from  a  given  network  architecture  can  be  modeled  in  this  way  as  a  network  of 
adaptive  e-machine  learners. 
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2.  Finite- Population  Dynamical  Systems 


Collectives,  in  any  practical  setting,  consist  of  finite  populations  of  agents.  Small,  here, 
is  measured  relative  to  how  sparsely  a  finite  population  coarse-grains  the  infinite-population 
state  space.  Finite  population  effects,  which  we  have  investigated  previously  in  evolutionary 
and  ecological  dynamics,  [95-98]  dominate  collective  behavior.  We  showed  that  these  effects 
can  have  significant,  even  counter-intuitive,  dynamical  consequences.  Our  studies  of  finite- 
population  effects  for  general  nonlinear  dynamical  systems  demonstrated  novel  behaviors 
that  arise  from  the  interaction  of  the  coarse-grained  state  space,  sampling  stochasticity,  and 
nonlinear  state-space  structures  (e.g.,  attractor  types,  stable-unstable  manifolds,  and  so  on). 


3.  Evolution  of  Finite-Memory  Agents 

An  important  type  of  collective  is  an  evolving  population  of  replicating  agents  that  modify 
and  transmit  their  internal  control  dynamics  to  others.  This  setting  is  useful  for  several 
reasons.  First,  one  can  use  evolutionary  methods  to  minimize  the  size  of  individual  control 
systems  or  to  optimize  communication  topologies.  Since  we  were  interested  in  the  simplest 
real-time  agents  that  are  functional,  typically  only  finite  memory  and  finite  time  are  available 
to  implement  a  decision  or  control  strategy.  For  this  reason  finite-memory  e-machines  and 
e-transducers  are  the  most  natural  representation  for  this  situation.  Given  this,  the  second 
reason  an  evolutionary  setting  is  useful  is  that  one  way  to  find  the  smallest  such  agents  is  to 
use  evolutionary  search  methods. 

We  had  developed  some  expertise  with  evolving  spatially  extended  (cellular) 
automata.  [99-101]  In  this  project  we  applied  similar  evolutionary  methods,  but  to  evolve 
finite-state  machines.  [102-107]  We  developed  a  theory  of  the  evolutionary  search  for  evolv¬ 
ing  e-machines  as  we  have  done  for  our  previous  studies  of  evolving  cellular  automata  and 
epochal  evolutionary  search  dynamics.  [108] 

4 ■  Self- Organizing  Agent  Population  Dynamics 

We  completed  our  first  projects  in  the  emergence  of  structural  complexity  in  groups 
of  interacting  finite-state  agents — the  finitary  process  soup.  One  result  is  that  evolutionary 
pressures  build  structure  hierarchically  and  prefer  relatively  less  complex  individuals  that  are 
generalists  and  can  participated  in  a  wider  range  of  interactions.  This  result  has  important 
implications  for  the  automated  design  of  control  systems.  [109,  110] 

We  extended  our  work  on  self-organization  in  groups  of  interacting  finite-state  agents 
to  spatially  distributed  agents.  We  showed  that  space  adds  a  new  and  important  kind  of 
information  storage  and  this,  in  turn,  significantly  affects  the  kinds  of  coordination  and 
structure  seen  in  agent  populations. 
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5.  Theory  and  Analysis  of  Emergent  Collective  Adaptation 


In  mathematical  terms,  agent  collectives  are  high-dimensional  stochastic  dynamical  sys¬ 
tems.  Though  they  may  hold  in  principle,  many  of  the  basic  techniques  and  concepts  from 
dynamical  systems  theory  do  not  directly  apply  to  agent  collectives  and  other  complex  sys¬ 
tems.  There  are  a  number  of  difficulties  that  confront  the  application  of  nonlinear  dynamics 
to  high-dimensional  systems  generally,  such  as  distinguishing  nonstationary  from  station¬ 
ary  and  transient  from  asymptotic  behaviors.  What  does  carry  over  from  the  study  of 
low- dimensional  dynamical  systems  is  the  fact  that  geometric  structures  in  state  space  still 
determine  the  available  information  processing  in  a  high-dimensional  system. 

How  can  distributed  systems  compute  robustly?  We  feel  there  is  some  basic  feature  that 
has  been  missed  to  date  that  explains  this  capacity  as  found  in  neurobiological  systems,  as  an 
important  and  instructive  example.  There,  the  observation  that  transient,  not  asymptotic, 
behavior  determines  the  largest  fraction  of  image  classification  and  recognition  in  vision  sug¬ 
gests  that  one  would  only  need  to  roughly  design  the  basin  boundaries  guiding  the  behavior 
of  a  neurobiological  system  in  order  to  effect  a  given  type  of  image  processing.  This  comes  at 
the  cost  of  accuracy  of  the  implementation — a  trade-off  that  would  have  to  be  studied.  In  any 
case,  we  believe  that  this  basic  question  can  be  elucidated  by  applying  existing  techniques  to 
high-dimensional  distributed  dynamical  systems  and  that  the  answer  will  delineate,  at  the 
same  time,  the  functionality  and  robustness  that  agent  collectives  can  manifest. 


6.  Persistent  Chaos  in  High  Dimensions 

To  explore  general  properties  of  collective  adaptation,  we  completed  the  central  theoretical 
results  and  simulation  analysis  on  high-dimensional  dynamical  systems  models  of  learning — 
recurrent  neural  networks.  [Ill]  We  found  that  instability,  in  the  form  of  deterministic  chaos, 
is  a  common  and  persistent  class  of  dynamical  behavior.  The  implications  for  multiagent 
systems,  which  are  necessarily  high  dimensional,  is  clear.  It  may  be  very  difficult  to  design 
away  instabilities  in  collective  behavior. 


7.  Dynamics  of  Midtiagent  Games 

We  completed  a  new  mathematical  framework — MultiAgent  Dynamical  Systems 
(MADS) — for  modeling  the  dynamics  of  multiagent  games.  We  established  new  results  on 
the  global  behavior  of  these  high- dimensional  dynamical  systems,  including  sufficient  con¬ 
ditions  for  stability  and  instability  and  limits  on  the  achievable  degrees  of  complexity  and 
of  emergent  coordination.  [112,  113]  We  extended  this  work  to  substantially  more  general 
methods  to  derive  equations  of  motion  from  any  type  of  local  adaptation  mechanism  used 
by  agents. 
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8.  Multiple  Agents  Servicing  Multiple  Tasks 


As  a  concrete  example  of  this,  we  reformulated  important  multiagent  systems  (MAS) 
notions  (“strategy”,  “adaptation",  “cooperation”)  in  purely  dynamical-systems  terms.  The 
goal  was  to  describe  and  predict  the  emergence  of  spontaneous  order  in  an  MAS  in  terms 
of  high-dimensional  bifurcation  theory.  In  particular,  we  developed  a  model  for  the  TASK 
Open  Experiment  Framework  (OEF)  problem,  what  we  called  Multiple  Agents  Servicing 
Multiple  Tasks  (MASMT)  and  completed  its  basic  theoretical  description.  We  developed  an 
interactive  graphical  interface  to  the  simulation  code  that  allows  users  to  explore  the  system’s 
parameter  space  and  its  nonlinear  dynamics.  We  also  developed  code  and  ran  simulations  to 
analyze  the  attractor-basin  structure  of  the  MASMT  dynamical  systems.  The  code  streams 
for  the  interactive  graphical  interface  to  the  MASMT  and  general  multiagent  dynamical 
systems  (MADS)  simulation  codes  were  merged. 


9.  Causal  Synchrony 

a.  Causal  Synchrony  in  Multiagent  Systems  We  developed  ways  for  measuring  and 
mapping  causal  synchrony  in  distributed  agent  systems.  In  particular,  we  collaborated  with 
Kristina  Lerman  (TASK  PI)  and  Aram  Galstyan,  of  the  University  of  Southern  California’s 
Information  Sciences  Institute,  to  measure  the  spatiotemporal  distribution  of  synchrony  in 
the  network  minority  game  and  determine  its  relation  to  the  emergence  of  coordination  and 
global  efficiency. 

b.  Neural  Causal  Synchrony  We  applied  computational  mechanics  to  measure  syn¬ 
chrony  and  coherence  in  heterogeneous  multiagent  networks.  For  the  initial  phase  of  develop¬ 
ing  and  then  testing  these  measures  on  well  characterized  network  systems,  we  collaborated 
with  a  computational  neuroscience  group  at  the  University  of  San  Francisco  on  applications 
to  neuronal  networks.  We  examined  extensive  simulations  of  biophysically  realistic  networks. 


C.  TASK  Demonstration  Project 

We  developed  an  autonomous  vehicle  demonstration  for  the  TASK  Open  Experiment 
Framework.  This  was  done  as  a  supplemental  effort  to  the  original  Dynamics  of  Learning 
project.  The  goal  was  to  demonstrate  coordination  and  adaptation  in  a  multiagent  system 
(MAS)  on  a  range  of  tasks  specified  in  the  Open  Experiment  Framework  problem. 

Specifically,  we  developed  a  robotic  platform — the  Robotic  MultiAgent  Development  Sys¬ 
tem  (RoMADS) — that  was  used  to  experimentally  validate  our  theoretical  predictions,  de¬ 
rived  from  computational  mechanics,  about  learning,  adaptation,  and  cooperation  in  multi¬ 
agent  systems.  We  constructed  a  collective  of  robotic  vehicles  and  a  high-efficiency  database 
system  that  permits  extensive,  detailed  analysis  of  their  behavior.  We  developed  a  set  of 
benchmark  learning  algorithms  for  individual  robots  and  the  robot  collective  including  rein¬ 
forcement  learning  algorithms. 
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1.  MAS  Dynamics  Simulator 


The  goal  here  was  to  develop  an  interactive  simulation  of  dynamical  models  of  MAS 
coordination  and  adaptation.  This  consisted  of  the  following  components. 

1.  Automatically  generate  symbolic  equations  of  motion  from  our  reinforcement-learning 
MAS  dynamics  framework. 

2.  Easily  integration  of  these  into  our  differential  equation  solvers. 

3.  An  interactive  GUI  that  guides  exploration  and  quantitative  analysis  of  emergent  be¬ 
haviors. 


2.  Robotic  MAS  Development  System  (RoMADS) 

The  goal  here  was  to  develop  a  test  bed  for  basic  MAS  behaviors  using  physical  vehi¬ 
cles.  RoMADS  was  developed  to  full  functionality  and  was  used  for  multiagent  behavioral 
experiments  and  design.  It  included  a  video  server  and  object-tracking  system,  web-based 
control  and  design  system,  and  a  high-performance  database  engine  for  data  (especially 
video)  storage  and  analysis. 

The  system  consisted  of  several  hardware  and  software  components: 

1.  RoboCars:  a  collection  of  small  robotic,  wirelessly  controlled  vehicles; 

2.  RoboCar  distributed  control  software; 

3.  The  GROUNDS  data  logging  and  messaging  system; 

4.  A  Linux  cluster  for  backend  analysis  and  real-time  simulation  of  learning  algorithms; 
and 

5.  An  WWW  interface  to  all  control  and  monitor  functions. 

Six  RoboCars  were  made  from  standard  Lego  Mindstorms  kits  with  additional  sensing 
hardware.  They  were  programmed  using  the  LeJOS  Java  operating  system  for  the  Mind- 
storms  “brick”  (embedded  controller).  The  interaction  arena  in  which  the  cars  ran  was  a  4’ 
x  8’  foot  experiment  table  with  predictable  surface  and  walls. 

The  RoboCars  had  two  differential  DC  drive  motors  that  provided  three  speeds  of  forward 
and  backward  motion  as  well  as  turn-in-place  direction  changing.  For  the  “native”  sensors 
they  had  four  bumpers  to  detect  collisions  and  proximity  to  other  cars,  a  light  sensor  used 
for  locating  other  vehicles  or  stationary  targets,  distance  and  compass  sensors  that  provided 
moderately  accurate  position  estimates,  and  an  internal  battery  voltage  indicator  to  trigger 
recharging  behavior. 

The  cars  accepted  simple  motion  commands  and  returned  telemetry  over  their  built-in 
infrared  link  which  was  connected  to  a  RoboCar  control  system  host  running  on  an  external 
PC  level  processor. 
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A  RoboCar  software  module  supplied  basic  intelligence  for  each  of  the  RoboCars.  The 
software,  written  in  pure  Java,  allowed  heterogeneous  devices  to  be  integrated  into  a  MAS 
and  then  controlled  and  tracked  in  a  common  manner.  It  also  provided  hierarchical  levels 
of  control,  sequencing,  and  adaptive  behavior  for  each  device,  as  well  as  access  to  all  data 
structures  for  storage,  analysis,  and  reloading.  Using  this  system  each  device  could  be 
returned  to  any  check-pointed  state  in  the  course  of  a  set  of  experiments.  Along  with 
managing  the  live  data,  all  commands  and  results  were  logged  to  local  hies  or  the  external 
GROUNDS  host  for  online  monitoring  and  offline  analysis. 

The  General  Robot  Observations  Using  Nonlinear  Dynamical  Systems  (GROUNDS)  sys¬ 
tem  was  hosted  by  a  J2EE  (Java  2  Enterprise  Edition)  system  that  was  distributed  across 
a  number  server  platforms  for  load  balancing  and  failover  as  necessary.  The  J2EE  system 
supplied  a  container  for  Java  Beans  (user  code  modules),  a  small  relational  database  (which 
integrates  with  any  JDBC  compliant  DBMS),  and  a  web  server.  Using  this  system  all  robot 
data  was  logged  to  the  database  and  retrieved  by  external  programs  using  standard  proto¬ 
cols  including  HTTP  (web),  RMI  (native  Java),  and  CORBA  (standard  C++).  GROUNDS 
also  acted  as  a  switching  center,  connecting  external  programs  directly  to  control  individual 
RoboCars.  The  switching  center  implemented  higher-level  communication  and  coordination 
behaviors  between  RoboCars  and  other  devices  in  a  controlled  manner.  As  a  side  effect,  it 
also  was  a  straightforward  matter  to  allow  external  servlets  access  to  the  RoboCars  to  run 
web  based  experiments.  Thus,  the  entire  system  was  made  open  to  (authorized)  use  from 
anywhere  on  the  net. 

The  Linux  cluster  was  used  to  develop  and  execute  the  analysis  and  metrics  software 
described  above,  for  the  RoMADS  system,  as  well  as  the  simulations. 

The  J2EE  web  server  executed  GROUNDS  servlets  that  provide  access  to  all  monitoring 
and  control  functions  over  the  Internet.  This  included  a  webcam  viewing  the  experiment 
table,  access  to  logged  data,  and  direct  access  to  the  RoboCars  as  needed. 


3.  RoMADS  Sensor  Nets 

The  goal  was  to  demonstrate  sensor  data  integration  techniques. 

We  integrated  MICA  Motes  into  the  RoMADS  systems  to  augment  vehicle  communication 
and  sensory  inputs,  both  on  the  RoboCars  and  in  intelligent  stationary  agents.  A  Mote  was 
yet  another  device  to  RoMADS  where  it  logged  data  to  GROUNDS.  The  data  was  then 
distributed  to  the  RoboCars  by  external  GROUNDS  programs. 


4 •  Open  Experiment  Framework  Related  Experiments 

The  goal  here  was  to  demonstrate  multiagent  behaviors  with  physical  vehicles  and  sensor 
nets  under  the  following  guidelines. 

1.  Stationary  and  mobile  targets:  Tracking  and  servicing  using  adaptive  algorithms. 

2.  Quantitative  analysis:  Collect  data  over  many  experiments  to  measure  MAS  efficiency, 
emergent  intelligence  (statistical  complexity),  and  coordination  (causal  synchrony). 
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3.  Develop  mathematical  dynamical  models  (MAS  reinforcement  learning  framework)  to 
predict  experimentally  observed  MAS  behaviors. 

4.  MAS  experiments: 

(a)  Compare  individual  and  group  behaviors  when  mapping  an  area. 

(b)  Compare  behaviors  in  simple  tracking  and  servicing  activities. 

(c)  Demonstrate  robustness  and  adaptability  to  a  change  in  the  number  of  targets  or 
agents. 

(d)  Provide  a  2+1/2  D  demonstration  of  system  integration,  using  a  second  level  on 
the  experiment  arena  to  simulate  aerial  surveillance  of  ground  vehicles. 

5.  System  Development  Status 

We  evaluated  the  Intel/UC  Berkeley  MICA2  Motes  and  their  Tiny  OS  software  for  in¬ 
tegration  into  our  distributed  robotic  system.  These  additions  were  geared  to  the  TASK 
Demonstration  Project. 

We  designed  a  new  sensor  I/O  board  for  the  MICA  Mote  controllers  and  fabricated  and 
tested  them.  They  were  integrated  into  the  RoboCars. 

We  designed  and  built  a  third  generation  of  the  RoboCar  platform  that  relies  completely 
on  the  MICA  Mote  controller.  The  new  version  was  smaller,  5”  diameter  compared  to  the  8” 
diameter  of  the  previous  version.  This  allows  doubling  the  number  of  robotic  agents  running 
simultaneously  within  the  limited  space  of  the  experimental  arena.  Equally  important,  the 
run-time  quadrupled  from  about  2  hours  to  8  hours.  Finally,  the  communication  bandwidth 
was  greatly  increased  from  a  single  9600  bps  serially  polled,  shared  channel  to  a  64  kps, 
packet-based  CDMA  channel.  This  substantially  facilitated  scaling  RoMADS  to  more  agents. 

One  remaining,  but  important  task  is  to  estimate  how  communication  speeds  scale  with 
increasingly  large  numbers  of  vehicles  sharing  the  radio  channel.  It  turned  out,  for  exam¬ 
ple,  that  the  MICA2  controller  communication  speeds  were  not  sufficiently  high  to  support 
extrapolated  future  needs.  This  led  us  to  upgrade  to  the  MICAZ  microcontroller  which  is 
based  on  ZigBee  250  kbps  radio. 

Along  these  lines,  the  PI  established  a  collaboration  with  Sun  Microsystems  on  its 
SunSpots  initiative  for  sensor  networks.  We  investigated  adapting  the  hardware  platform  for 
our  Multiagent  Development  System  to  the  all-Java-based  controllers  that  Sun  Microsystems 
Labs  has  pioneered  as  the  basis  of  a  fourth  generation  of  RoboCar.  The  benefits  include 
seamless  integration  into  the  MADS  (almost  all-)  Java-based  system  and  another  factor  of  2 
(estimated)  in  communication  bandwidth.  The  main  drawback  is  that  this  controller  is  still 
underdevelopment  and  not  yet  commercially  available. 


6.  RoMADS  Experiments  on  Multiagent  Systems 

At  the  TASK  PI  meetings  we  reported  encouraging  results  on  an  algorithm  for  tab¬ 
ula  rasa  learning  running  on  robotic  vehicles.  These  showed  that  individual  robots  and 
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robot  collectives  can  learn  specific  useful  behaviors — such  as,  wall  following  and  collision 
avoidance — without  these  behaviors  being  programmed  in  by  hand. 

We  completed  empirical  tests  of  one  of  our  tabula  rasa  learning  algorithms  on  our  experi¬ 
mental  multiagent  robotic  platform.  We  experimented  with  learning  in  groups  of  two,  three, 
and  four  robotic  vehicles  in  experimental  arenas  of  varying  geometry.  We  quantitatively 
characterized  the  previously  observed  behaviors  that  (i)  learning  takes  longer  in  groups  due 
to  the  increased  complexity  of  the  statistics  seen  by  each  agent,  (ii)  nonetheless  the  robots 
eventually  learn  basic  navigational  skills,  and  (iii)  they  learn  new  behaviors  that  allow  them 
to  avoid  crowding. 


V.  CONCLUSIONS 

The  main  goals  of  this  project  were  to  understand  the  dynamics  of  learning  in  single  agents 
well  enough  so  that  we  could  analyze  and  design  agent  collectives  that  perform  a  desired 
global  task.  Our  approach  to  these  goals  was  to  extend  the  computational  mechanics  frame¬ 
work  in  a  way  that  provided  a  dynamical  model  of  learning  and  adaptation — a  model  that 
could  be  analyzed  quantitatively  and  theoretically.  We  also  used  computational  mechanics 
to  develop  methods  to  monitor  the  emergence  of  coordinated  behavior  in  agent  collectives 
and  used  these  to  test  and  verify  global  functionality,  or  lack  of  this  functionality,  and  to 
compare  distributed  cooperation  to  alternative  approaches  to  performing  a  given  task. 

The  main  advantages  of  the  approach  are  its  grounding  in  a  rigorous  theory  of  how 
nonlinear  processes  intrinsically  compute  (i.e.,  its  grounding  in  computational  mechanics) 
and  the  attention  it  pays  to  well  developed  theories  of  emergent  collective  behavior,  such  as 
found  in  pattern  formation  theory  and  statistical  mechanics.  We  believe  that  the  project 
laid  the  foundation  for  a  new  and  novel  view  of  agent  collectives  and  of  the  issues  involved 
in  their  design.  The  main  disadvantages  would  be  that  in  trying  to  develop  predictive 
analytical  theories  for  the  behavior  of  learning  and  adaptive  systems  we  may  not  be  able  to 
solve  them  in  closed-form.  In  fact,  we  showed  that  their  collective  behaviors  are  extremely 
rich — exhibiting  the  panoply  of  dynamical  behaviors  that  one  now  expects  from  large-scale, 
complex  dynamical  systems. 
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APPENDIX  A:  SPONSORED  WORKSHOPS 


A  number  of  workshops  were  organized  by  the  project  as  part  of  its  research  and  outreach 
efforts. 

1.  The  PI  co-organized  the  founding  workshop  for  his  SFI  Network  Dynamics  Program, 
“Structure  and  Dynamics  in  Complex  Adaptive  Networks”,  Santa  Fe,  New  Mexico, 
10-12  August  2000. 

2.  The  PI  co-organized  the  SFI  Business  Network  Meeting  on  “Network  Dynamics” ,  Santa 
Fe,  New  Mexico,  22  March  2001. 

3.  The  PI  co-organized  the  workshop  “The  Internet  as  a  Complex  Adaptive  System”, 
Santa  Fe  Institute,  Santa  Fe,  New  Mexico,  28-30  March  2001. 

4.  The  PI  and  Stephanie  Forrest  (UNM,  TASK  PI)  helped  organize  the  April  2001  TASK 
PI  Meeting  in  Santa  Fe,  New  Mexico,  and  at  SFI. 

5.  The  PI  organized  organized  the  October  2002  TASK  PI  Meeting  in  Santa  Fe,  New 
Mexico,  and  at  SFI.  This  included  the  PI  and  his  group  presenting  a  half-day  tutorial 
on  the  Dynamics  of  Learning  project  and  computational  mechanics,  statistical  physics, 
and  information-theoretic  methods  for  multiagent  systems. 

6.  The  PI  co-organized  a  successful  workshop  on  “Collective  Cognition:  Mathemati¬ 
cal  Foundations  of  Distributed  Intelligence,”  bringing  together  workers  in  statistical 
physics,  mathematical  learning  theory,  computer  science,  economics,  sociology,  and 
other  fields  that  are  concerned  with  distributed  information  processing  and  cognition. 
(For  a  complete  list  of  participants,  talk  titles  and  abstracts,  and  other  information 
on  the  workshop,  see  http://cse.ucdavis.edu/~dynlearn/colcog.)  The  workshop 
addressed  key  TASK  issues  of  distributed  control,  adaptation  in  heterogeneous  envi¬ 
ronments,  and  spontaneous  leadership. 

APPENDIX  B:  PUBLICATIONS  AND  REPORTS 

1.  “When  Evolution  is  Revolution — Origins  of  Innovation”,  in  Evolutionary  Dynamics — 
Exploring  the  Interplay  of  Selection,  Neutrality,  Accident,  and  Function,  James  P. 
Crutchfield  and  P.  Schuster,  editors,  Santa  Fe  Institute  Series  in  the  Science  of  Com¬ 
plexity,  Oxford  LIniversity  Press,  New  York  (2001)  104-138. 

2.  “Dynamics  of  Evolutionary  Processes”,  in  Evolutionary  Dynamics — Exploring  the  In¬ 
terplay  of  Selection,  Neutrality,  Accident,  and  Function,  James  P.  Crutchfield  and 
P.  Schuster,  editors,  Santa  Fe  Institute  Series  in  the  Science  of  Complexity,  Oxford 
University  Press,  New  York  (2001)  1-22.  (James  P.  Crutchfield  and  P.  Schuster). 
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3.  James  P.  Crutchfield  and  Peter  Schuster,  editors,  “Evolutionary  Dynamics — Exploring 
the  Interplay  of  Selection,  Neutrality,  Accident,  and  Function”,  Oxford  University 
Press,  New  York  (2001). 

4.  James  P.  Crutchfield  and  David  P.  Feldman,  “Synchronizing  to  the  Environ¬ 
ment:  Information  Theoretic  Constraints  on  Agent  Learning”,  Advanced  in  Com¬ 
plex  Systems  (2001)  submitted.  Santa  Fe  Institute  Working  Paper  01-03-020  and 
arXiv.org/abs  /  nlin.  AO /0103038. 

5.  Wim  Horclijk,  Cosma  Rohilla  Shalizi  and  James  P.  Crutchfield,  “An  Upper  Bound  on 
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29.  Cosma  R.  Shalizi,  “Causal  Architecture  of  Collectives,”  presentation  at  Collective  Cog¬ 
nition  workshop,  21-25  January  2002. 
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48.  Michael  Schippling,  “An  Overview  of  Distributed  Robotics”,  Dynamics  of  Learning 
Group  seminar,  29  October  2002;  see  presentation  materials  on  Dynamics  of  Learning 
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56.  Dave  Albers  and  Yuzuru  Sato,  “Multiagent  Dynamical  Systems:  Review  and  Applica¬ 
tion  to  the  Multiple  Agents  Servicing  Multiple  Tasks  Problem,”  Dynamics  of  Learning 
Seminar,  27  May  2003,  Santa  Fe  Institute. 

57.  James  P.  Crutchfield,  “Dynamics  of  Multiagent  Systems”,  BISON  Consortium  Meet¬ 
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58.  James  P.  Crutchfield,  “An  Introduction  to  Computational  Mechanics”,  SFI  Complex 
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59.  Yuzuru  Sato,  “Dominance  of  Milnor  Attractors  in  Globally  Coupled  Dynamical  Sys¬ 
tems”,  Dynamics  of  Learning  Seminar,  15  July  2003. 

60.  Dave  Albers,  “Structural  Stability  and  Partial  Hyperbolicity  in  Large  Dynamical  Sys¬ 
tems”,  Dynamics  of  Learning  Seminar,  22  July  2003. 

61.  Erik  Talvitie,  “Problems  in  Inferring  the  Structure  of  Space”,  Dynamics  of  Learning 
Seminar,  29  July  2003;  SFI  REU  Talk,  8  August  2003. 

62.  Selah  Lynch,  “Algorithms  for  Multisymbol  epsilon-Machine  Reconstruction”,  Dynam¬ 
ics  of  Learning  Seminar,  12  August  2003;  SFI  RELI  Talk,  13  August  2003. 

63.  John  Albers,  “Multiple  Agents  Servicing  Multiple  Tasks”,  Dynamics  of  Learning  Sem¬ 
inar,  26  August  2003;  SFI  REU  Talk,  27  August  2003. 

64.  Olof  Gornerup,  “Finitary  Process  Soup”,  Dynamics  of  Learning  Seminar,  2  September 
2003,  Santa  Fe  Institute. 

65.  James  P.  Crutchfield,  “Automated  Pattern  Discovery”,  Invited  Plenary  Lecture,  SIAM 
Meeting  on  Industrial  Applications  of  Complex  Systems,  Toronto,  13-15  October  2003. 

66.  James  P.  Crutchfield,  “Dynamical  Embodiments  of  Computation  in  Cognition”,  Ron 
Brachman  (IPTO  Director)  et  al  visit,  Santa  Fe  Institute,  Santa  Fe,  New  Mexico,  22-23 
October  2003. 

67.  James  P.  Crutchfield,  “Complex  Systems  Theory?” ,  Invited  Lecture,  SFI  Faculty  Re¬ 
treat,  Bishop’s  Lodge,  Santa  Fe,  New  Mexico,  24-25  October  2003. 

68.  James  P.  Crutchfield,  “Intrinsic  Computation”,  Physics  Department  Colloquium,  Uni- 
versity  of  California,  Davis,  12  November  2003. 

69.  James  P.  Crutchfield,  “Automated  Pattern  Discovery”,  Engineering  Department  Sem¬ 
inar,  LIniversity  of  California,  Davis,  13  November  2003. 


27 


70.  James  P.  Crutchfield,  “Form  and  Function:  From  pattern  to  semantics  and  on  to  func¬ 
tion1',  Collegium  Budapest-SFI  Workshop  on  Form  and  Function,  Budapest,  Hungary, 
16-19  November  2003. 

71.  James  P.  Crutchfield,  “Dynamics  of  Learning  in  Distributed  Robotics”,  TASK  PI 
Meeting,  Washington  DC,  4-5  December  2003. 

72.  James  P.  Crutchfield,  “Intrinsic  Computation  and  Pattern  Discovery”,  Colloquium, 
Physics  Department,  University  of  New  Mexico,  Albuquerque,  12  January  2004. 

73.  Nihat  Ay  (SFI),  “Information  Geometry,  Part  I”,  Dynamics  of  Learning  Group,  Sem¬ 
inar,  Santa  Fe  Institute,  20  January  2004. 

74.  Nihat  Ay  (SFI),  “Information  Geometry,  Part  II”,  Dynamics  of  Learning  Group,  Sem¬ 
inar,  Santa  Fe  Institute,  20  January  2004. 

75.  Yuzuru  Sato  (SFI),  “Multiagent  Dynamical  Systems  and  Learning”,  Arizona  Days 
workshop,  Los  Alamos  National  Laboratory,  30  January  2004. 

76.  Chris  Strelioff  (SFI/UIUC),  “Statistical  Error  Estimates  for  Markov  Models  of  Dynam¬ 
ical  Systems”,  Dynamics  of  Learning  Group,  Seminar,  Santa  Fe  Institute,  10  February 
2004. 

77.  Yuzuru  Sato  (SFI),  “Multiagent  Dynamical  Systems  and  Learning”,  Research  Seminar, 
Santa  Fe  Institute,  17  February  2004. 

78.  Dave  Albers  (SFI/Madison),  “Multiple  Agents  Servicing  Multiple  Tasks,  Part  I”,  Dy¬ 
namics  of  Learning  Group,  Seminar,  Santa  Fe  Institute,  24  February  2004. 

79.  Nihat  Ay  (SFI),  “Causal  State  Reduction  of  Complex  Processes”,  Research  Seminar, 
Santa  Fe  Institute,  25  February  2004. 

80.  James  P.  Crutchfield,  “Intrinsic  Computation  and  Nanotechnology”,  invited  lecture, 
Conference  on  Nanotechnology,  Biotechnology,  Information  Technology,  and  Cognitive 
Science,  New  York,  26  February  2004. 

81.  Dave  Albers  (SFI/Madison),  “Multiple  Agents  Servicing  Multiple  Tasks,  Part  II”, 
Dynamics  of  Learning  Group,  Seminar,  Santa  Fe  Institute,  2  March  2004. 

82.  Nihat  Ay  (SFI),  “Information  Geometry,  Part  III”,  Dynamics  of  Learning  Group, 
Seminar,  Santa  Fe  Institute,  23  March  2004. 

83.  James  P.  Crutchfield,  “Theory  Theory”,  invited  lecture,  Seminar  on  Computation  in 
Natural  Systems,  Center  for  Nonlinear  Studies,  Los  Alamos  National  Laboratory,  29 
March  2004. 

84.  Nihat  Ay  (SFI),  “Information  Geometry,  Part  IV”,  Dynamics  of  Learning  Group, 
Seminar,  Santa  Fe  Institute,  30  March  2004. 
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86.  James  P.  Crutchfield,  “Pattern  Discovery  and  Automated  Theory  Building”,  Physics- 
Astronomy  Colloquium,  Northwestern  University,  Chicago,  Illinois,  9  April  2004. 

87.  James  P.  Crutchfield  (SFI),  “Complex  Systems  Theory?”,  Science  Board  Spring  Meet¬ 
ing,  Santa  Fe  Institute,  7  May  2004. 

88.  Olof  Gornerup  (SFI),  “The  Emergence  of  Hierarchical  Structures  in  Evolution:  The 
Finitary  Process  Soup”,  Dynamics  of  Learning  Group,  Seminar,  Santa  Fe  Institute,  1 
June  2004. 

89.  James  P.  Crutchfield,  Dynamics  of  Learning  Group  (SFI),  “Overview  of  the  Dynamics 
of  Learning:  Introductions  to  the  RELT  Summer  Interns” ,  Dynamics  of  Learning  Group, 
Seminar,  Santa  Fe  Institute,  8  June  2004. 

90.  Dave  Albers  (SFI/Madison),  “Outstanding  Problems  in  the  Statistical  Mechanics  of 
Nonequilibrium”,  Dynamics  of  Learning  Group,  Seminar,  Santa  Fe  Institute,  22  June 
2004. 

91.  John  Albers  (SFI/Madison),  “Adjoint  Sensitivity  Analysis”,  Dynamics  of  Learning 
Group,  Seminar,  Santa  Fe  Institute,  29  June  2004. 

92.  James  P.  Crutchfield,  “Multi Agent  Dynamical  Systems:  Completion  of  the  Theoretical 
Framework  and  Simulator”,  TASK  PI  Meeting,  Washington,  DC,  5  August  2004. 

93.  James  P.  Crutchfield  and  Michael  Schippling,  “RoMADS — Robotic  MultiAgent  Devel¬ 
opment  System”  and  “MultiAgent  Dynamical  Systems” ,  TASK  Demonstration,  Wash¬ 
ington,  DC,  4  August  2004. 

94.  David  Albers,  “Robust  Chaos  in  High-Dimensional  Dynamical  Systems”,  Ph.D.  Dis¬ 
sertation  Defense,  Physics  Department,  University  of  Wisconsin,  Madison,  18  August 
2004. 

95.  Carl  McTague  (SFI),  “Automated  Pattern  Discovery — Constructing  Multi- Regular 
Language  Filters”,  Dynamics  of  Learning  Group,  Seminar,  Santa  Fe  Institute,  17  Au¬ 
gust  2004. 

96.  James  P.  Crutchfield,  “Pattern  and  Pattern  Discovery — A  Review  of  Computation 
Mechanics”,  Redwood  Neurosciences  Institute,  Menlo  Park,  CA,  2  September  2004. 

97.  James  P.  Crutchfield,  “Cellular  Automata  Computational  Mechanics — Patterns  and 
Discovery”,  Keynote  speech,  ACRI  2004,  Amsterdam,  Netherlands,  26  October  2004. 

98.  James  P.  Crutchfield,  “The  Evolution  of  Structural  Complexity”,  Center  for  Liv¬ 
ing  Technology  and  Statistics  Department,  Universite  de  Ca  Forsca,  Venice,  Italy, 
2  November  2004. 
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99.  James  P.  Crutchfield,  “Chaos  and  Complexity”,  Graduate  Seminar,  Venice  Interna¬ 
tional  Program  of  the  University  of  Virginia,  Venice,  Italy,  4  November  2004. 

100.  James  P.  Crutchfield,  “Objects  that  Make  Objects:  The  Population  Dynamics  of  Struc¬ 
tural  Complexity”,  Mathematical  Biology  Seminar,  UC  Davis,  15  February  2005. 

101.  James  P.  Crutchfield,  “Pattern  Discovery  for  Spatial  Processes”,  Sun  Microsystems 
Laboratory  2005  Open  House,  Computer  Museum,  Palo  Alto,  California,  27  April 
2005. 

102.  James  P.  Crutchfield,  “Multiagent  Dynamical  Systems”,  Seminar  Series  in  Complex 
Systems,  Environmental  Sciences  Department,  UC  Davis,  2  June  2005. 

103.  James  P.  Crutchfield,  “Multiagent  Dynamical  Systems”,  Sun  Microsystems  Research 
Laboratories,  Mountain  View,  California,  17  August  2005. 

104.  James  P.  Crutchfield,  “Multiagent  Dynamical  Systems”,  Evolution  and  Economics 
Seminar,  University  of  California,  Davis,  25  October  2005. 

105.  James  P.  Crutchfield,  Keynote  speech  on  “Frontiers  in  Complex  Systems”,  Hackers 
2005,  Santa  Cruz,  California,  13  November  2005. 

106.  James  P.  Crutchfield,  “Objects  that  Make  Objects:  The  Population  Dynamics  of  Struc¬ 
tural  Complexity”,  Workshop  on  the  Evolution  of  Biological  Complexity,  Ohio  State 
University,  Columbus,  Ohio,  17  November  2005. 

107.  James  P.  Crutchfield,  “Structure,  Meaning,  and  Function:  A  Dynamical  Systems  Per¬ 
spective”,  International  School  on  Semiotic  Dynamics,  Language,  and  Complexity,  Et- 
tore  Majorana  Foundationa  and  Centre  for  Scientific  Culture  Erice,  Italy,  14  December 
2005. 

APPENDIX  D:  PRESENTATIONS  HOSTED  BY  THE  PROJECT 

1.  Martin  Nilsson  (LANL),  “Time  Series  Modeling  and  Sensor  Fusion”,  Dynamics  of 
Learning  Group  seminar,  24  September  2002. 

2.  Lukasz  Debowski,  “Inhnitary  Processes  and  Linguistics”,  Dynamics  of  Learning  Group 
seminar,  6  August  2002. 

3.  Scott  and  Jena  Page  (Economics,  University  of  Michigan),  “Problem  Solving  in 
Groups”,  Dynamics  of  Learning  Group  seminar,  16  July  2002. 

4.  Eric  Smith,  “Self-Organization  as  Structural  Refrigeration”,  Dynamics  of  Learning 
group  seminar,  19  and  26  November  2002. 

5.  John  Miller  (Carnegie-Mcllon),  “Coordination  and  Competition”,  Dynamics  of  Learn¬ 
ing  Group  seminar,  18  March  2003. 
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6.  Sam  Bowles  (Amherst/SFI)  and  Jessica  Flack  “Social  Cooperation  in  Chimps  and 
Economies),  Dynamics  of  Learning  Group  seminar,  21  January  2003. 

7.  Koichi  Fujimoto,  “How  Fast  Elements  can  Affect  Slow  Dynamics” ,  Dynamics  of  Learn¬ 
ing  Seminar,  8  July  2003. 

8.  Matt  Tanner  and  Eric  Smith,  “How  do  Smart  Agents  Use  Their  Internal  Knowledge 
for  Effective  Group  Interactions?”,  Dynamics  of  Learning  Seminar,  5  August  2003. 

9.  Geoff  Canright  and  Kenth  Engo,  “Biologically  Inspired  Self-Organization  in  Dynamical 
Networks” ,  Dynamics  of  Learning  Seminar,  22  August  2003. 

10.  Chi-Chi  May,  “Biological  Coding  Theory”,  Dynamics  of  Learning  Seminar,  30  Septem¬ 
ber  2003. 

11.  Emily  Stone  (Utah  State),  “Dynamic  models  of  PCR”,  Dynamics  of  Learning  Group, 
Seminar,  Santa  Fe  Institute,  5  March  2004. 

12.  Eizo  Akiyama  (Tokyo  Institute  of  Technology) ,  “Game  Dynamical  Systems”,  Dynamics 
of  Learning  Group,  Seminar,  Santa  Fe  Institute,  9  March  2004. 

13.  Garrett  Kenyon  (LANL),  “The  Neurodynamics  of  Pattern  Recognition  in  Early  Vi¬ 
sion”  ,  Dynamics  of  Learning  Group,  Seminar,  Santa  Fe  Institute,  13  April  2004. 

14.  David  Peak  (Utah  State),  “Distributed  Computation  in  Plant  Respiration”,  Collo¬ 
quium,  Santa  Fe  Institute,  16  April  2004. 

15.  Tony  Bell  (Redwood  Neuroscience  Institute),  “Topics  in  the  Dynamics  of  Neurolearn¬ 
ing”,  Dynamics  of  Learning  Group,  Seminar,  Santa  Fe  Institute,  27  April  2004. 

16.  James  Theiler  (LANL),  “Machine  Learning  Approaches  to  Modeling  Dynamics  Sys¬ 
tems”,  Dynamics  of  Learning  Group,  Seminar,  Santa  Fe  Institute,  5  May  2004. 

17.  Luis  Bettencourt  (LANL),  “In  vitro  Neurodynamics  and  Distributed  Biosensors”,  Dy¬ 
namics  of  Learning  Group,  Seminar,  Santa  Fe  Institute,  11  May  2004. 

18.  Frantisek  Matus  (University  of  Prague),  “Information  Geometry  and  Inference”,  Dy¬ 
namics  of  Learning  Group,  Seminar,  Santa  Fe  Institute,  25  May  2004. 

19.  Ken  Burns  (Northwestern),  “Dynamical  Systems  Tools  in  High  Dimensions”,  Dynamics 
of  Learning  Group,  Seminar,  Santa  Fe  Institute,  24  August  2004. 
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